Dungeons and Data: A Large-Scale NetHack Dataset

Recent breakthroughs in the development of agents to solve challenging sequential decision making problems such as Go [50], StarCraft [58], or DOTA [3], have relied on both simulated environments and large-scale datasets. However, progress on this research has been hindered by the scarcity of open-sourced datasets and the prohibitive computational cost to work with them. Here we present the NetHack Learning Dataset (NLD), a large and highly-scalable dataset of trajectories from the popular game of NetHack, which is both extremely challenging for current methods and very fast to run [23]. NLD consists of three parts: 10 billion state transitions from 1.5 million human trajectories collected on the NAO public NetHack server from 2009 to 2020; 3 billion state-action-score transitions from 100,000 trajectories collected from the symbolic bot winner of the NetHack Challenge 2021; and, accompanying code for users to record, load and stream any collection of such trajectories in a highly compressed form. We evaluate a wide range of existing algorithms including online and offline RL, as well as learning from demonstrations, showing that significant research advances are needed to fully leverage large-scale datasets for challenging sequential decision making tasks

MiniHack the Planet: A Sandbox for Open-Ended Reinforcement Learning Research

Progress in deep reinforcement learning (RL) is heavily driven by the availability of challenging benchmarks used for training agents. However, benchmarks that are widely adopted by the community are not explicitly designed for evaluating specific capabilities of RL methods. While there exist environments for assessing particular open problems in RL (such as exploration, transfer learning, unsupervised environment design, or even language-assisted RL), it is generally difficult to extend these to richer, more complex environments once research goes beyond proof-of-concept results. We present MiniHack, a powerful sandbox framework for easily designing novel RL environments. MiniHack is a one-stop shop for RL experiments with environments ranging from small rooms to complex, procedurally generated worlds. By leveraging the full set of entities and environment dynamics from NetHack, one of the richest grid-based video games, MiniHack allows designing custom RL testbeds that are fast and convenient to use. With this sandbox framework, novel environments can be designed easily, either using a human-readable description language or a simple Python interface. In addition to a variety of RL tasks and baselines, MiniHack can wrap existing RL benchmarks and provide ways to seamlessly add additional complexity

GPflux: A Library for Deep Gaussian Processes

We introduce GPflux, a Python library for Bayesian deep learning with a strong emphasis on deep Gaussian processes (DGPs). Implementing DGPs is a challenging endeavour due to the various mathematical subtleties that arise when dealing with multivariate Gaussian distributions and the complex bookkeeping of indices. To date, there are no actively maintained, open-sourced and extendable libraries available that support research activities in this area. GPflux aims to fill this gap by providing a library with state-of-the-art DGP algorithms, as well as building blocks for implementing novel Bayesian and GP-based hierarchical models and inference schemes. GPflux is compatible with and built on top of the Keras deep learning eco-system. This enables practitioners to leverage tools from the deep learning community for building and training customised Bayesian models, and create hierarchical models that consist of Bayesian and standard neural network layers in a single coherent framework. GPflux relies on GPflow for most of its GP objects and operations, which makes it an efficient, modular and extensible library, while having a lean codebase

Insights from the NeurIPS 2021 NetHack Challenge

In this report, we summarize the takeaways from the first NeurIPS 2021 NetHack Challenge. Participants were tasked with developing a program or agent that can win (i.e., ‘ascend’ in) the popular dungeon-crawler game of NetHack by interacting with the NetHack Learning Environment (NLE), a scalable, procedurally generated, and challenging Gym environment for reinforcement learning (RL). The challenge showcased community-driven progress in AI with many diverse approaches significantly beating the previously best results on NetHack. Furthermore, it served as a direct comparison between neural (e.g., deep RL) and symbolic AI, as well as hybrid systems, demonstrating that on NetHack symbolic bots currently outperform deep RL by a large margin. Lastly, no agent got close to winning the game, illustrating NetHack’s suitability as a long-term benchmark for AI research

Induced pseudoscalar coupling of the proton weak interaction

The induced pseudoscalar coupling $g_p$ is the least well known of the weak coupling constants of the proton's charged--current interaction. Its size is dictated by chiral symmetry arguments, and its measurement represents an important test of quantum chromodynamics at low energies. During the past decade a large body of new data relevant to the coupling $g_p$ has been accumulated. This data includes measurements of radiative and non radiative muon capture on targets ranging from hydrogen and few--nucleon systems to complex nuclei. Herein the authors review the theoretical underpinnings of $g_p$, the experimental studies of $g_p$, and the procedures and uncertainties in extracting the coupling from data. Current puzzles are highlighted and future opportunities are discussed.Comment: 58 pages, Latex, Revtex4, prepared for Reviews of Modern Physic

Lamb shifts and hyperfine structure in 6Li+ and 7Li+: Theory and experiment

High-precision laser-resonance measurements accurate to ±0.5 MHz or better are reported for transitions among the 1s2s 3S1-1s2p 3PJ hyperfine manifolds for each of J=0, 1, and 2 in both Li+6 and Li+7. A detailed analysis of hyperfine structure is performed for both the S and P states, using newly calculated values for the magnetic dipole and electric quadrupole coupling constants, and the hyperfine shifts subtracted from the measurements. The resulting transition frequencies are then analyzed on three different levels. First, the isotope shifts in the fine-structure splittings are calculated from the relativistic reduced mass and recoil terms in the Breit interaction, and compared with experiment at the ±0.5-MHz level of accuracy. This comparison is particularly significant because J-independent theoretical uncertainties reduce through cancellation to the ±0.01-MHz level. Second, the isotope shifts in the full transition frequencies are used to deduce the difference in rms nuclear radii. The result is Rrms(6Li)-Rrms(7Li)=0.15±0.01 fm, in agreement with nuclear scattering data, but with substantially improved accuracy. Third, high-precision calculations of the low-order non-QED contributions to the transition frequencies are subtracted from the measurements to obtain the residual QED shifts. The isotope-averaged and spin-averaged effective shift for Li+7 is 37 429.40±0.39 MHz, with an additional uncertainty of ±1.5 MHz due to finite nuclear size corrections. The accuracy of 11 parts per million is the best two-electron Lamb shift measurement in the literature, and is comparable to the accuracies achieved in hydrogen. Theoretical contributions to the two-electron Lamb shift are discussed, including terms of order (αZ)4 recently obtained by Chen, Cheng, and Johnson [Phys. Rev. A 47, 3692 (1993)], and the results used to extract a QED shift for the 2 3S1 state. The result of 30 254±12 MHz is shown to be in good accord with theory (30 250±30 MHz) when two-electron corrections to the Bethe logarithm are taken into account by a 1/Z expansion method. © 1994 The American Physical Society